Big Data

This second post in a short series on TV news innumeracy looks at an even more ludicrous example – the notion of “bellwether” counties. The pundits seem to have an obsession with counties – or towns, or even precincts – that have voted for the winning side in several consecutive elections, as if that information actually meant anything. Well, it doesn’t, and the reason it doesn’t has implications for financial analysis.

At a dinner party last night, one of the guests posed a question: Imagine a roomful of people chosen at random. How many people need to be in the room for there to be at least a 50% probability that at least two of the people have the same birthday? The answer to this puzzle tells us a lot about how we process risk & uncertainty, and how we should think when we build financial models.

A widely cited quote asserts that big data is like teenage sex, because there’s less of it going on than is widely believed, as well as lots of other misinformation about the practice. It’s an apt comment, and I have my own reasons for arguing that big data is like teenage sex.

At a recent IMA seminar, I had the opportunity to sit in on several excellent presentations. One of them was Toby Groves’s overview of big data, a powerful software tool that has rightfully gotten much attention, but also has inherent limitations. It sometimes takes real wisdom and willpower to see when we should STOP using big data.

In one of my last 2014 posts, I lambasted a particularly silly Yahoo! Finance blog titled “This Signal Has a Perfect Record of Forecasting Year-End Gains,” in which the author used a convoluted calculation based on the Market Volatility Index (VIX) to predict the S&P 500’s performance for the remainder of the year. We rarely have the opportunity to skewer such pontificating so promptly, but this “signal” no longer has a perfect record.

I try to be positive. I really do. But yesterday I came across a post that is so ridiculous that I can’t stay silent. The writer observes that in every year since the Chicago Board Options Exchange’s Market Volatility Index (^VIX) was created in 1986, if the VIX traded at more than 34% above its one-month low at least once during the month of December, the S&P 500 showed a gain from that point until the end of the year. (Got that? There will be a quiz on this later.) The writer claims to have found a predictor “whose results have been so overwhelmingly consistent that it is hard to ignore.”

Today’s USA Today story about last night’s World Series game (GO GIANTS!) includes an interesting graphic, showing the success of “wild card” teams since they were first included in Major League Baseball’s postseason playoffs in 1995. 2014 is the first year that two wild card teams have faced off in the World Series – yes, a major-league record!!!

A fascinating piece in a recent Washington Post draws a striking comparison between income inequality and political polarization. It suggests a high correlation between greater inequality and more polarization. That’s all well and good, but it’s all meaningless if you don’t hypothesize a causal relationship between the two.